#backend dev without much scaling is surprisingly easy these days

[mourning-again-in-america]

Very funny to me that one of the best pitches (if true) for any webshit framework is "what if webdev were good"

[t-baba]

A new JS runtime, ESLint 7, npm's future, and more

#488 — May 15, 2020

Unsubscribe  :  Read on the Web

JavaScript Weekly

Deno 1.0 Released — Two years ago, Ryan Dahl, the creator of Node.js, gave a talk about the 10 things he regretted about Node.js. At the same time, he introduced Deno, a prototype of a new, security-first, npm-less JavaScript runtime — now it's considered ready for the big time and, at a minimum, is worth having a quick play with. If you want to do a walkthrough tutorial to see if it’s to your taste or not, this is a good one by Flavio Copes.

Ryan Dahl, Bert Belder, and Bartek Iwańczuk

Psst.. we launched Deno Weekly as a way to stay up to date with Deno. We'll only feature it from time to time in JavaScript Weekly.

JavaScript Features To Forget — It’d be easy to say.. that's just, like, your opinion, man.. but when it’s the author of about 73 (slight exaggeration) JavaScript books including JavaScript: The Definitive Guide, it’s worth listening.

David Flanagan

Don’t Build Auth From Scratch. Focus On Your App — Spend less time on authentication and authorization and more time developing your awesome app. Auth built for <devs>. Download our community edition for free.

FusionAuth sponsor

Playwright 1.0: Fast and Reliable Cross-Browser Testing — We first featured Playwright, Microsoft’s take on a cross-browser alternative to Puppeteer, a few months ago, but it’s already at 1.0. This post does a good job on selling it.

Arjun Attam (Microsoft)

Announcing TypeScript 3.9 — You know it's JavaScript plus syntax for type declarations and annotations by now, right? 3.9 gains a variety of editor improvements, performance improvements, and tweaks to inference and Promise.all. No awaited just yet though.

Daniel Rosenwasser (Microsoft)

Recoil: An Exprimental State Management Library for React — Interesting not only because it comes from Facebook, but because there’s a pretty good 23 minute talk about it and it follows the latest React standards.

Facebook

⚡️ Quick bytes:

WebAssembly is taking us into interesting places.. such as writing PONG clones in COBOL that we can play in the browser. The COBOL code is pretty nice, to be fair.

JSNation Live is the latest in a line of online JavaScript conferences, this one coming on June 18-19 and, curiously, with both free and paid options.

Kite has unveiled AI powered JavaScript 'completions' based upon deep learning over 22 million JavaScript files.

💻 Jobs

Senior Software Engineer — Save Lives & Make an Impact — We use Node/TS/React & ML to provide crisis support via SMS. Help us scale globally with a focus on privacy and security.

Crisis Text Line

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

📚 Tutorials and Opinions

▶  Tom Preston-Werner Talks About Redwood.js — GitHub co-founder Tom Preston-Werner talks about Redwood.js, a new full-stack JavaScript framework built around React, GraphQL, and Prisma.

Full Stack Radio podcast

Standard IO: Under the Hood — What really happens behind the scenes when you call console.log?

Vladimir Keleshev

An Initial Introduction to npm v7 — If you thought the npm blog would fade away after the GitHub acquisition, you thought wrong! Isaac Schlueter is back with a status update on what the next major version of npm will offer and is promising a series of posts to come going into more detail. We'll probably focus more on this in Node Weekly in coming weeks.

The npm Blog

Now Up to 4GB of Memory in WebAssembly Applications on V8 — It’s 4GB because WebAssembly currently uses 32 bit pointers but till now WebAssembly apps in Chrome and Emscripten have been limited to 2GB for.. reasons.

Andreas Haas, Jakob Kummerow, and Alon Zakai

Connect Users to Vital Business Data with NodeRun (Webinar On‑Demand)

Profound Logic sponsor

Why We Made a New Component Library Instead of Buying One — It principally came down to the company involved preferring to have total control but they also felt Vue.js was well suited to the task.

Alessandro Grosselle

Second-Guessing the Modern Web — What if everyone’s wrong? Can we solve things in a better way that single page applications? Interesting thought piece and Rich Harris replied with In Defense of the Modern Web.

Tom MacWright

John Conway's FRACTRAN, A Ridiculous, Yet Surprisingly Deep Language — A beautiful JavaScript-heavy tribute to John Conway (of Game of Life fame) by trying to reimplement his esoteric FRACTRAN language.

Reg Braithwaite

What Every Developer Should Know About TCP — You’re every developer! And so am I!

Roberto Vitillo

🔧 Code & Tools

ESLint v7.0.0 Released — The popular pluggable and configurable linter tool for identifying and reporting on patterns in your code. Node 8 support is dropped.

ESLint

A Much Faster Way to Debug Code Than with Breakpoints or console.log — Move forward and backwards through your code to understand what led to a specific bug, view runtime values, edit-and-continue, and more.

Wallaby.js sponsor

Next.js 9.4 Released: The React Powered Site Building Framework — Just two months after 9.3 (which was already an exciting release) comes the even more significant 9.4 which includes Fast Refresh, a new “instantaneous” hot reloading experience, plus incremental static generation where static pages can be generated or re-rendered in the background as traffic comes in.

Vercel

Fuse.js 6.0: Lightweight Fuzzy-Search Without Dependencies — Want a simple search feature without a dedicated backend? This could help. It’s been around for years but 6.0.0 has just dropped with more new goodies like logical query operators. - GitHub repo.

Kiro Risk

Perfume.js 5.0: A Web Performance Library for Measuring User-Centric Perf Metrics — Only 2KB when gzipped, supports the latest browser performance APIs for precise metrics on things like first paint, first input delay, total blocking time, etc.

Leonardo Zizzamia

Shifty: A Teeny Tiny Tweening Engine — All it does is tweening. It’s a low level animation solution that you can integrate into any rendering mechanism of your choice. The examples here demonstrate it well as it can be used for ‘animating’ things in an unconventional sense. GitHub repo.

Jeremy Kahn

MongoDB Is Easy. Now Make It Powerful. Free Download for 30 Days. — Using MongoDB Atlas? Studio 3T is the professional GUI and IDE that unlocks the power you need.

Studio 3T sponsor

Color2K: A Color Parsing and Manipulation Library in 2KB or Less — A key goal here was to be a lot smaller than the competition “while still satisfying all of your color manipulation needs in an sRGB space”.

Rico Kahler

by via JavaScript Weekly https://ift.tt/2WzGfUB

[iyarpage]

How development for AWS changes the local setup

When a new project is set up these days in the cloud this usually means that AWS is used, that the backend is split into multiple services (e.g. micro services), the frontend and backend communicate using REST and multiple managed AWS services are used. Such a setup allows to easily leverage many services AWS provides and reduces the effort and time to implement the desired functionality. To fully use this potential the development process has to be reviewed and adjusted.

In the old days a single monolith was developed locally on the developer computer. Dependencies like database, message queue or SMTP server were mostly installed locally or, if this was not possible, mocked or a central installation was used. In most cases it was possible that the developer could be certain that if the application runs on his computer it will also run on other environments (e.g. test stage or production environment). This approach is not feasible with how applications are developed these days. The differences stem from multiple sources:

Splitting the application into multiple parts (e.g. micro services) creates a fully distributed application in which the glue which connects the parts is becoming more important. One side effect is for example that the glue also has to be tested. Another consequence of a distributed application is that the application becomes unwieldy in the way that multiple programs have to be running to experience the full application. The development team also has to think about distributed failures and aspects like resilience are getting important.

Using cloud services introduces new APIs and new way of thinking. So for example the database is not just there but you also have to think about request capacities for your database access.

Using AWS especially adds the burden of not being able to install the cloud solution on your local computer(s). Other solutions like Kubernetes allow you to do this. While not a real limitation these days you lose the ability to develop offline. More important is that the development gets more expensive since these AWS services are to be paid also for the developers and this sometimes mean that the cost will scale with the number of developers.

DevOps can solve the first two issues but the last one could be described as vendor lock-in.

Let’s take a look at some scenarios and assess them. In the following “stage” means a self contained version of the software under development. This could be (for AWS): a set of EC2 instances, an ECS cluster, a unique URL, a set of DynamoDB tables identified by a prefix, a set of CodePipelines and other AWS services.

Use a shared dev stage

In a very easy setup there is one stage for development. If you want to do a change you commit and push your change, wait for the build and deployment and then test your change. While it’s easy to get this setup running it has certain drawbacks:

The long turnaround times since you have to wait for the build and deployment. This takes usually a few minutes.

Debugging your services is cumbersome or impossible.

Since AWS CodePipelines do not support branches the commit has to be performed in the master. This may be ok if the team develops in the master using feature toggles. This however prevents other teams to use feature branches and perform code reviews using pull requests.

Potential problems because of conflicts between developers which use the same code areas and the same stage.

Use personalized stages

One idea to reduce the drawbacks it to create a stage for each developer. This also means the code pipelines have to be duplicated and adjusted to listen to developer specific branch names. This allows to review the code with pull requests in addition to developing independently. This is especially important if there are incompatible code changes in the backend services or changes in the state itself. However there are some existing drawbacks and a new one:

Long turnaround times

Debugging is hard or impossible

Extra costs for each stage

Run application services locally

To improve the turnaround times and allow to debug the application services easily it is possible to run the application services locally. This may mean that only the one service under development is started or that all services are running locally. While it’s easy to run the applicationservices locally it’s not that easy or even impossible to get the AWS services (i.e. SQS, S3) running on the local machine. So the AWS services of one stage have to be used. While this setup can be combined with one shared dev stage or with the personalized stages mentioned above the shared dev stage has two disadvantages:

If there are structural changes to the AWS services or the data stored / transferred extra effort is required to ensure that other developers using the same stage are unaffected. It may not be worth the effort and a personalized stage is the better solution.

There may be too little control over the execution: if during the test one application service adds a message to a queue (e.g. SQS) and another locally running application service is the consumer of this queue it is not guaranteed that the locally running consumer will get this message.

Another aspect is how these application services are reached:

Accessed by the frontend or other application services: This is usually through a load balancer. In such a case a locally running load balancer replacement is required. We have successfully used nginx for this with a script which converts AWS load balancer rules into a nginx config file.

Activated by AWS services: An example here is SQS consumption. This is a problem with a shared stage but works for personalized stages.

In general this approach is possible and is recommended if the conditions are right (low frequency of structure changes, either no AWS service activation or personalized stages are used).

Running as much as possible local

It is possible to run more, in addition to the load balancer, AWS services locally. This allows to further reduce the dependency for AWS services during development and reduce the costs. Besides DynamoDB which is available for local installation the other AWS services are not available for local installation. One can view a replacement software as locally running AWS service or one can also see it as a mock for the real AWS service. This also depends on how sophisticated the service implementation has to be to successfully run the application on the local computer. In general it also means that the fact that the application works locally doesn’t mean necessarily that it also works on a full AWS stage.

There are replacements for multiple AWS services:

moto: Originally a mocking library but also allows to start as a separate server.

localstack: “A fully functional local AWS cloud stack. Develop and test your cloud apps offline!”

Eucalyptus (doc): complete private cloud implementation which claims to be API compatible with AWS.

Out of these three Eucalyptus looks not active maintained. It also looks like the owner of the backing company changed recently. LocalStack uses moto but aims to provide a more complete solution. There is also the beginning of a commercial company backing LocalStack.

For specific services (an older list):

For Kinesis:

kinesalite: “An implementation of Amazon’s Kinesis built on LevelDB”

For S3:

minio: “Minio is an open source object storage server with Amazon S3 compatible API. “ It supports multiple underlying storage providers.

ceph: “Ceph is a unified, distributed storage system designed for excellent performance, reliability and scalability.�� It also offers an S3 interface.

Fake S3: “A lightweight server clone of Amazon S3 that simulates most of the commands supported by S3 with minimal dependencies”

SQS:

elasticmq: “Message queueing system with an actor-based Scala and Amazon SQS-compatible interfaces”

goaws: “AWS (SQS/SNS) Clone for Development testing”

Lambda and API gateway:

serverless-offline: “Emulate AWS λ and API Gateway locally when developing your Serverless project”

emulambda: “Python emulator for AWS Lambda”

Conclusion

There are surprisingly few experience reports about local AWS development. LocalStack on the other side was developed by Atlassian and while there is no official press release it is reasonable to assume that they felt a problem and developed a solution for it.

If money is not a concern the recommendation is to provide a personalized stage for each developer and run the application services locally. In such a case a stripped down stage without the active application services is enough.

If the number of developers is larger or a stage consists of many costly elements the recommendation is to try to use replacements which run locally, for example LocalStack. If there is a setup which avoids AWS services you may also want to consider using this setup in CI/CD. Still plan for the case that the real AWS services may behave differently.

The post How development for AWS changes the local setup appeared first on codecentric AG Blog.

How development for AWS changes the local setup published first on https://medium.com/@koresol

[mobilenamic]

How development for AWS changes the local setup

When a new project is set up these days in the cloud this usually means that AWS is used, that the backend is split into multiple services (e.g. micro services), the frontend and backend communicate using REST and multiple managed AWS services are used. Such a setup allows to easily leverage many services AWS provides and reduces the effort and time to implement the desired functionality. To fully use this potential the development process has to be reviewed and adjusted.

In the old days a single monolith was developed locally on the developer computer. Dependencies like database, message queue or SMTP server were mostly installed locally or, if this was not possible, mocked or a central installation was used. In most cases it was possible that the developer could be certain that if the application runs on his computer it will also run on other environments (e.g. test stage or production environment). This approach is not feasible with how applications are developed these days. The differences stem from multiple sources:

Splitting the application into multiple parts (e.g. micro services) creates a fully distributed application in which the glue which connects the parts is becoming more important. One side effect is for example that the glue also has to be tested. Another consequence of a distributed application is that the application becomes unwieldy in the way that multiple programs have to be running to experience the full application. The development team also has to think about distributed failures and aspects like resilience are getting important.

Using cloud services introduces new APIs and new way of thinking. So for example the database is not just there but you also have to think about request capacities for your database access.

Using AWS especially adds the burden of not being able to install the cloud solution on your local computer(s). Other solutions like Kubernetes allow you to do this. While not a real limitation these days you lose the ability to develop offline. More important is that the development gets more expensive since these AWS services are to be paid also for the developers and this sometimes mean that the cost will scale with the number of developers.

DevOps can solve the first two issues but the last one could be described as vendor lock-in.

Let’s take a look at some scenarios and assess them. In the following “stage” means a self contained version of the software under development. This could be (for AWS): a set of EC2 instances, an ECS cluster, a unique URL, a set of DynamoDB tables identified by a prefix, a set of CodePipelines and other AWS services.

Use a shared dev stage

In a very easy setup there is one stage for development. If you want to do a change you commit and push your change, wait for the build and deployment and then test your change. While it’s easy to get this setup running it has certain drawbacks:

The long turnaround times since you have to wait for the build and deployment. This takes usually a few minutes.

Debugging your services is cumbersome or impossible.

Since AWS CodePipelines do not support branches the commit has to be performed in the master. This may be ok if the team develops in the master using feature toggles. This however prevents other teams to use feature branches and perform code reviews using pull requests.

Potential problems because of conflicts between developers which use the same code areas and the same stage.

Use personalized stages

One idea to reduce the drawbacks it to create a stage for each developer. This also means the code pipelines have to be duplicated and adjusted to listen to developer specific branch names. This allows to review the code with pull requests in addition to developing independently. This is especially important if there are incompatible code changes in the backend services or changes in the state itself. However there are some existing drawbacks and a new one:

Long turnaround times

Debugging is hard or impossible

Extra costs for each stage

Run application services locally

To improve the turnaround times and allow to debug the application services easily it is possible to run the application services locally. This may mean that only the one service under development is started or that all services are running locally. While it’s easy to run the applicationservices locally it’s not that easy or even impossible to get the AWS services (i.e. SQS, S3) running on the local machine. So the AWS services of one stage have to be used. While this setup can be combined with one shared dev stage or with the personalized stages mentioned above the shared dev stage has two disadvantages:

If there are structural changes to the AWS services or the data stored / transferred extra effort is required to ensure that other developers using the same stage are unaffected. It may not be worth the effort and a personalized stage is the better solution.

There may be too little control over the execution: if during the test one application service adds a message to a queue (e.g. SQS) and another locally running application service is the consumer of this queue it is not guaranteed that the locally running consumer will get this message.

Another aspect is how these application services are reached:

Accessed by the frontend or other application services: This is usually through a load balancer. In such a case a locally running load balancer replacement is required. We have successfully used nginx for this with a script which converts AWS load balancer rules into a nginx config file.

Activated by AWS services: An example here is SQS consumption. This is a problem with a shared stage but works for personalized stages.

In general this approach is possible and is recommended if the conditions are right (low frequency of structure changes, either no AWS service activation or personalized stages are used).

Running as much as possible local

It is possible to run more, in addition to the load balancer, AWS services locally. This allows to further reduce the dependency for AWS services during development and reduce the costs. Besides DynamoDB which is available for local installation the other AWS services are not available for local installation. One can view a replacement software as locally running AWS service or one can also see it as a mock for the real AWS service. This also depends on how sophisticated the service implementation has to be to successfully run the application on the local computer. In general it also means that the fact that the application works locally doesn’t mean necessarily that it also works on a full AWS stage.

There are replacements for multiple AWS services:

moto: Originally a mocking library but also allows to start as a separate server.

localstack: “A fully functional local AWS cloud stack. Develop and test your cloud apps offline!”

Eucalyptus (doc): complete private cloud implementation which claims to be API compatible with AWS.

Out of these three Eucalyptus looks not active maintained. It also looks like the owner of the backing company changed recently. LocalStack uses moto but aims to provide a more complete solution. There is also the beginning of a commercial company backing LocalStack.

For specific services (an older list):

For Kinesis:

kinesalite: “An implementation of Amazon’s Kinesis built on LevelDB”

For S3:

minio: “Minio is an open source object storage server with Amazon S3 compatible API. “ It supports multiple underlying storage providers.

ceph: “Ceph is a unified, distributed storage system designed for excellent performance, reliability and scalability.” It also offers an S3 interface.

Fake S3: “A lightweight server clone of Amazon S3 that simulates most of the commands supported by S3 with minimal dependencies”

SQS:

elasticmq: “Message queueing system with an actor-based Scala and Amazon SQS-compatible interfaces”

goaws: “AWS (SQS/SNS) Clone for Development testing”

Lambda and API gateway:

serverless-offline: “Emulate AWS λ and API Gateway locally when developing your Serverless project”

emulambda: “Python emulator for AWS Lambda”

Conclusion

There are surprisingly few experience reports about local AWS development. LocalStack on the other side was developed by Atlassian and while there is no official press release it is reasonable to assume that they felt a problem and developed a solution for it.

If money is not a concern the recommendation is to provide a personalized stage for each developer and run the application services locally. In such a case a stripped down stage without the active application services is enough.

If the number of developers is larger or a stage consists of many costly elements the recommendation is to try to use replacements which run locally, for example LocalStack. If there is a setup which avoids AWS services you may also want to consider using this setup in CI/CD. Still plan for the case that the real AWS services may behave differently.

The post How development for AWS changes the local setup appeared first on codecentric AG Blog.

How development for AWS changes the local setup published first on https://medium.com/@TheTruthSpy